Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

disableCompression: Expose configuration to toggle Envoy GZIP compression on the responses #6546

Open
wants to merge 26 commits into
base: main
Choose a base branch
from

Conversation

chaosbox
Copy link

@chaosbox chaosbox commented Jul 9, 2024

For #6511

This PR adds,

  • Listener configuration that exposes a boolean flag to disable compression, by default compression is enabled. This also provides us a way to disable if the users prefer to trade network for CPU, especially when teams want to run lean Envoy instances and rely on horizontal scalability.
  • We will run the test build for a while in our cluster to show the actual cost benefit.

Related #310, there had been mentions about disabling compression, the ticket we had raised shows the reason where disabling compression can bring cost benefits.

@chaosbox chaosbox requested a review from a team as a code owner July 9, 2024 09:04
@chaosbox chaosbox requested review from skriss and sunjayBhatia and removed request for a team July 9, 2024 09:04
@sunjayBhatia sunjayBhatia requested review from a team, rajatvig and davinci26 and removed request for a team July 9, 2024 09:18
Copy link

github-actions bot commented Jul 9, 2024

Hi @chaosbox! Welcome to our community and thank you for opening your first Pull Request. Someone will review it soon. Thank you for committing to making Contour better. You can also join us on our mailing list and in our channel in the Kubernetes Slack Workspace

dependabot bot and others added 2 commits July 9, 2024 10:42
…jectcontour#6543)

Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](docker/setup-buildx-action@d70bba7...4fd8129)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: chaosbox <[email protected]>
Copy link

The Contour project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 14d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, the PR is closed

You can:

  • Ensure your PR is passing all CI checks. PRs that are fully green are more likely to be reviewed. If you are having trouble with CI checks, reach out to the #contour channel in the Kubernetes Slack workspace.
  • Mark this PR as fresh by commenting or pushing a commit
  • Close this PR
  • Offer to help out with triage

Please send feedback to the #contour channel in the Kubernetes Slack

@github-actions github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 25, 2024
@chaosbox
Copy link
Author

Ping

@github-actions github-actions bot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 29, 2024
@tsaarni tsaarni added the release-note/small A small change that needs one line of explanation in the release notes. label Aug 2, 2024
# Conflicts:
#	.github/workflows/build_main.yaml
#	.github/workflows/build_tag.yaml
#	.github/workflows/prbuild.yaml
Copy link
Member

@tsaarni tsaarni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've included suggestions inline for some linter nags (link).

Please add the new disableCompression option in "Configuration File" chapter in site/content/docs/main/configuration.md.

Please add also a changelog file.

internal/featuretests/v3/compression_test.go Outdated Show resolved Hide resolved
internal/featuretests/v3/compression_test.go Outdated Show resolved Hide resolved
test/e2e/httpproxy/envoy_compression_test.go Outdated Show resolved Hide resolved
test/e2e/httpproxy/envoy_compression_test.go Outdated Show resolved Hide resolved
test/e2e/httpproxy/envoy_compression_test.go Outdated Show resolved Hide resolved
internal/envoy/v3/listener.go Show resolved Hide resolved
@geomacy
Copy link

geomacy commented Aug 19, 2024

Thanks very much for the feedback @tsaarni 🙇 we'll have a look at this and try to get back with updates as soon as possible.

@geomacy
Copy link

geomacy commented Aug 21, 2024

Many thanks for taking the time to provide the lint fixes. We'll update the docs and add the changelog as soon as possible.

Copy link

codecov bot commented Aug 21, 2024

Codecov Report

Attention: Patch coverage is 70.12987% with 23 lines in your changes missing coverage. Please review.

Project coverage is 80.97%. Comparing base (54ceade) to head (45ea946).
Report is 47 commits behind head on main.

Files with missing lines Patch % Lines
cmd/contour/servecontext.go 12.50% 13 Missing and 1 partial ⚠️
pkg/config/parameters.go 0.00% 7 Missing and 1 partial ⚠️
cmd/contour/serve.go 50.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #6546      +/-   ##
==========================================
- Coverage   81.76%   80.97%   -0.79%     
==========================================
  Files         133      133              
  Lines       15944    20052    +4108     
==========================================
+ Hits        13037    16238    +3201     
- Misses       2614     3519     +905     
- Partials      293      295       +2     
Files with missing lines Coverage Δ
internal/envoy/v3/listener.go 98.54% <100.00%> (+0.08%) ⬆️
internal/xdscache/v3/listener.go 92.05% <100.00%> (-0.07%) ⬇️
cmd/contour/serve.go 21.81% <50.00%> (-0.86%) ⬇️
pkg/config/parameters.go 86.00% <0.00%> (-2.03%) ⬇️
cmd/contour/servecontext.go 82.05% <12.50%> (-3.90%) ⬇️

... and 122 files with indirect coverage changes

---- 🚨 Try these New Features:

@geomacy geomacy force-pushed the main branch 3 times, most recently from 619ac16 to 16f72d7 Compare August 22, 2024 16:38
Geoff Macartney and others added 5 commits August 22, 2024 17:46
Co-authored-by: Tero Saarni <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Co-authored-by: Tero Saarni <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Co-authored-by: Tero Saarni <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Co-authored-by: Tero Saarni <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Co-authored-by: Tero Saarni <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
@geomacy
Copy link

geomacy commented Oct 9, 2024

Hi @davinci26 @tsaarni I have updated the branch according to your comment above, so that it is now possible to request different types of compression, including none. Rather than a separate nocompression flag, I've just provided an additional compression option "disabled".

The supported types of compression are those included in go-control-plane/envoy/extensions/compression, namely

  • gzip
  • brotli
  • zstd

This can be configured with the flag --compression or in config.

In-cluster tests

Tests against a deployment of "podinfo" on an in-house cluster.

out-of-the-box default behaviour

Ask for gzip encoding and gunzip the output:

~ [56]$ curl -v -H 'Accept-Encoding: gzip,deflate'  $URL/env | gzip -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 08 Oct 2024 13:41:14 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: gzip
< vary: Accept-Encoding
< server: envoy
<
{ [283 bytes data]
100   293    0   293    0     0   1421      0 --:--:-- --:--:-- --:--:--  1422
...
[
  "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  "HOSTNAME=podinfo-9f86b555d-c6cz7",
  "PODINFO_UI_COLOR=#34577c",

Note the content-encoding: gzip in the response - default behaviour is to gzip responses in response to the accept encoding request header.

disabled

set the flag in config (we hold values for the config in a configmap):

apiVersion: v1
data:
  contour.yaml: |-
    compression: disabled
    accesslog-format: json
...

Repeat the command above but this time should get text back and not need to un-gzip:

~ $ curl -v -H 'Accept-Encoding: gzip,deflate'  $URL/env
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 08 Oct 2024 13:50:11 GMT
< content-length: 814
< x-envoy-upstream-service-time: 1
< server: envoy
<
[
  "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  "HOSTNAME=podinfo-9f86b555d-c6cz7",
  "PODINFO_UI_COLOR=#34577c",
  "PODINFO_SERVICE_HOST=192.168.243.43",

No content-encoding: gzip in the response. Compression has been disabled.

brotli

Request brotli encoding on the response and decode it with the brotli CLI tool:

~ $ curl -v -H 'Accept-Encoding: br,deflate'  $URL/env | brotli -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 08 Oct 2024 13:53:45 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: br
< vary: Accept-Encoding
< server: envoy
<
{ [287 bytes data]
100   288    0   288    0     0   1020      0 --:--:-- --:--:-- --:--:--  1021
...
[
  "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  "HOSTNAME=podinfo-9f86b555d-c6cz7",
  "PODINFO_UI_COLOR=#34577c",

Note content-encoding: br in the response headers and the successful decode of the response. Brotli encoding was successfully requested.

zstd

Request zstd encoding on the response and decode it with the zstd CLI tool:

~ $ curl -v -H 'Accept-Encoding: zstd,deflate'  $URL/env | zstd -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 08 Oct 2024 14:03:27 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: zstd
< vary: Accept-Encoding
< server: envoy
<
{ [302 bytes data]
100   302    0   302    0     0   1089      0 --:--:-- --:--:-- --:--:--  1090
...
[
  "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  "HOSTNAME=podinfo-9f86b555d-c6cz7",

Note the content-encoding: zstd in the response headers and the successful decompression.
zstd encoding was successfully requested.

Invalid setting

Test the validation on the compression setting in contour config. Set an invalid value for the compression flag and restart contour pods. Pods fail to start with a simple error message recording the invalid value:

~ $ kubectl --context sky-pre-dev-euwest1 -n contour-main-external logs  contour-contour-6fc5b659c5-22d5g
time="2024-10-08T14:07:11Z" level=info msg="maxprocs: Leaving GOMAXPROCS=16: CPU quota undefined"
contour: error: invalid Contour configuration: invalid compression type: "bogus", try --help

@geomacy
Copy link

geomacy commented Oct 9, 2024

Could you kindly review and enable the build workflows on this to see if they all pass? In the branch where it was built I was getting an odd error on the upgrade tests, as noted here, and I wasn't sure if that might be because that was just a branch.

The upgrade tests are failing with

./test/scripts/kind-load-contour-image.sh
fatal: No names found, cannot describe anything.
CONTOUR_UPGRADE_FROM_VERSION= \
	CONTOUR_E2E_IMAGE=ghcr.io/projectcontour/contour:7e843b7a \
	go run github.com/onsi/ginkgo/v2/ginkgo -tags=e2e -mod=readonly -randomize-all -poll-progress-after=300s -v ./test/e2e/upgrade

Not 100% sure why that is, possibly something to do with actions/checkout on a branch? I think this change is sufficiently
advanced to make it worth merging to chaosbox/main, and working on resolving any futher test failures there.

Copy link
Contributor

@davinci26 davinci26 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some small comments but I think this is in the correct direction

// Values: `gzip` (default), `brotli`, `zstd`, `disabled`.
// Setting this to `disabled` will make Envoy skip "Accept-Encoding: gzip,deflate" request header and always return uncompressed response
// +optional
Compression EnvoyCompressionType `json:"compression,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use // +kubebuilder:validation:Enum=gzip,brotli,zstd,disabled as well?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 sounds good, I will add that

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @davinci26 added this in f7b261b

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Example:

$ kubectl apply -f - <<-EOF
> apiVersion: projectcontour.io/v1alpha1
kind: ContourConfiguration
metadata:
  name: example-contour-config-1
spec:
  envoy:
    listener:
      compression: bogus
EOF
The ContourConfiguration "example-contour-config-1" is invalid: spec.envoy.listener.compression: Unsupported value: "bogus": supported values: "gzip", "brotli", "zstd", "disabled"

// Values: `gzip` (default), `brotli`, `zstd`, `disabled`.
// Setting this to `disabled` will make Envoy skip "Accept-Encoding: gzip,deflate" request header and always return uncompressed response
// +optional
Compression EnvoyCompressionType `json:"compression,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API here is fairly constrained so we might want to think through how would we extend it if anyone comes up with the feature request to tune the API on different compression levels

I think adding another field called Compression Strategy might be fine or create a struct that right now has a single object

Compression *EnvoyCompression `json:"compression,omitempty"`
Member

type  EnvoyCompression struct {
Aglorithm Algorithm `json:"algorithm,omitempty"
}

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did think (briefly) about making the configurability greater, and/or having a wider range of compression algorithms. In the end I decided against it, at least for this PR -

  • while one could add more compression algorithms I thought that exposing the ones from envoy/extensions was a nice and easily explained choice. There are a couple of others in contrib/envoy/extensions but gzip/br/zstd seems like the ones that are most likely to be widely wanted. And I certainly didn't want to start thinking about making the algorithm choice be more "pluggable"!
  • as for configurability of algorithms I thought it wasn't something I felt I wanted to start making decisions on - I expect contour project members would perhaps not want to expose all the configuration options for every algorithm, but then this leaves you with the question of how to sensibly define some configuration that supports enough configurability for the different algorithms. It felt like a bit of a rabbit hole and I thought it would be something kept separate from this PR - better to try to keep changes as small as possible, and this one is meaty enough already.

Believe it or not I actually took a bit of inspiration from the Contour Philosophy 😆 (I did read around on how to contribute), particularly the bit on Sensible Defaults. I felt these kind of gave permission to simply offer a choice of a few widely used algorithms, using default settings. While a strategy field could be added here, it would be something of a no-op feature unless we took the time to implement at least basic configurability in it. I feel this would be best left to another PR. Would that be OK?

Copy link

@geomacy geomacy Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also now that I think of it, the feature here of most interest to us in our project is the compression disabling, so from an admittedly selfish point of view I would like to leave the configurability to a separate bit of work. Like everyone else we're very busy, and I have ended up having to do most of this work in my own time in the evenings and weekends :-(

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be transparent I dont want to design how we extend it but I am trying to think through if we leave enough space to extend the api.

In the end I decided against it, at least for this PR

My only issue is that if we go with the approach here and in 3 months we wanted to add another compression knob then we have two options:

  • Make a breaking change which might be fine due to the fact that this is a v1alpha1 package.
  • Add the option topline object in type EnvoyListenerConfig struct

If we think that these are fine options I think the api is good. I am just proponent of adding another struct that has a single member so we can extend just that struct

Like everyone else we're very busy, and I have ended up having to do most of this work in my own time in the evenings and weekends :-(

Yeah, a lot of the people contributing to this project to this into this capacity so I feel you.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will defer to @projectcontour/maintainers

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah now that you have laid it out like that I see what you mean. Let me take a look at it, hopefully this wouldn't be too large a change.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @davinci26 I have taken a stab at this in 5dcdc33, could you have a look and see what you think?

Copy link

@geomacy geomacy Oct 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still need to test this, I see some failures in e2e and will want also to run it on a local cluster, will try to get that done this week and add results in a comment.

@@ -471,6 +473,8 @@ func (s *Server) doServe() error {
SocketOptions: contourConfiguration.Envoy.Listener.SocketOptions,
}

s.log.WithField("context", "listenerConfig").Infof("compression setting: %s", listenerConfig.Compression)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this from debugging?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually left this in here deliberately - I think it's quite nice to be able to see the choice of algorithm reflected in the initial log output. But if that's not a typical contour thing to do I'll take it out. Let me know what would be preferred.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

took this out in b68f0b1

Geoff Macartney added 2 commits October 19, 2024 12:41
Signed-off-by: Geoff Macartney <[email protected]>
Copy link
Contributor

@davinci26 davinci26 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I have one api comment that I left open but I think the code overall lgtm and useful.

@@ -0,0 +1 @@
Add "compression" flag to set/disable the compression type applied in the default HTTP filter chain.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this might need update as well based on the new api

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

included in 5dcdc33

Geoff Macartney added 3 commits October 23, 2024 17:59
Signed-off-by: Geoff Macartney <[email protected]>
Avoid NPE in parameters.Validate

also tidy flag handling in registerServe, don't set compression struct unless parameter is supplied.

Signed-off-by: Geoff Macartney <[email protected]>
@geomacy
Copy link

geomacy commented Oct 25, 2024

Have added one more fix of a test, but I still need to finish testing on-cluster, will update the ticket when I get that completed.

Geoff Macartney added 5 commits October 28, 2024 10:10
Signed-off-by: Geoff Macartney <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
@geomacy
Copy link

geomacy commented Oct 29, 2024

Hello @davinci26 @tsaarni, I have updated the code per the proposal to support better API extensibility by wrapping the settings in a struct. I believe this PR is ready for review. Some results below from testing on a cluster:

default

out of the box behaviour

curl -v -H "Accept-Encoding: gzip,deflate"  $URL | gzip -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:36:04 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: gzip
< vary: Accept-Encoding
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

response was gzip.

parameter disabled

added to container args:

        - --compression=disabled

curl:

$ curl -v -H "Accept-Encoding: gzip,deflate"  $URL
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:40:55 GMT
< content-length: 353
< x-envoy-upstream-service-time: 1
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

No compression applied, it has been disabled.

parameter gzip

set container args to have

        - --compression=gzip

curl

$ curl -v -H "Accept-Encoding: gzip,deflate"  $URL | gzip -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:44:05 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: gzip
< vary: Accept-Encoding
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

parameter brotli

set container args to have

        - --compression=brotli

curl

$ curl -v -H "Accept-Encoding: br,deflate"  $URL | brotli -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:46:35 GMT
< x-envoy-upstream-service-time: 2
< content-encoding: br
< vary: Accept-Encoding
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

brotli encoding applied

parameter zstd

set container args with

        - --compression=zstd

curl

$ curl -v -H "Accept-Encoding: zstd,deflate"  $URL | zstd -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:49:23 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: zstd
< vary: Accept-Encoding
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

zstd encoding applied

config disabled

Remove compression flag from container args and set config map to have:

apiVersion: v1
data:
  contour.yaml: |-
    compression:
      algorithm: disabled
    ...

Restart contour pods. Do curl requesting gzip

$ curl -v -H "Accept-Encoding: gzip,deflate"  $URL
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:54:23 GMT
< content-length: 353
< x-envoy-upstream-service-time: 1
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",
  "version": "6.0.0",

compression is disabled.

config gzip

edit config

apiVersion: v1
data:
  contour.yaml: |-
    compression:
      algorithm: gzip
    ...

Restart contour and curl:

$ curl -v -H "Accept-Encoding: gzip,deflate"  $URL | gzip -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:56:37 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: gzip
< vary: Accept-Encoding
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

response encoded with gzip

config brotli

edit config to

apiVersion: v1
data:
  contour.yaml: |-
    compression:
      algorithm: brotli

Restart contour and curl:

$ curl -v -H "Accept-Encoding: br,deflate"  $URL | brotli -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:59:48 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: br
< vary: Accept-Encoding
< server: envoy
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

config zstd

edit config to

apiVersion: v1
data:
  contour.yaml: |-
    compression:
      algorithm: zstd

curl:

$ curl -v -H "Accept-Encoding: zstd,deflate"  $URL | zstd -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 10:29:20 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: zstd
< vary: Accept-Encoding
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

zstd compression applied.

@geomacy
Copy link

geomacy commented Nov 5, 2024

Hello @tsaarni, @davinci26, would you be able to review this? Thanks.

@davinci26
Copy link
Contributor

@geomacy will take a look later today.

@geomacy
Copy link

geomacy commented Nov 13, 2024

hi @davinci26 @tsaarni did you get a chance to look at this?

Copy link
Member

@tsaarni tsaarni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the long delay! I've finally had a chance to review and added comments inline.

The PR title and description are slightly outdated after the latest changes, and could use an update.

Thank you for your efforts and patience!

serve.Flag("accesslog-format", "Format for Envoy access logs.").PlaceHolder("<envoy|json>").StringVar((*string)(&ctx.Config.AccessLogFormat))

serve.Flag("compression", "Set or disable compression type in default Listener filters.").PlaceHolder("<gzip|brotli|zstd|disabled>").StringVar((*string)(&ctx.Config.Compression.Algorithm))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove the command line option and make configuration available only through config file and ContourConfiguration CRD.

The command line options currently are a bit of a mess, so we previously agreed to avoid introducing new options there unless absolutely necessary.

Comment on lines +32 to +39
func (a CompressionAlgorithm) Validate() error {
switch a {
case BrotliCompression, DisabledCompression, GzipCompression, ZstdCompression:
return nil
default:
return fmt.Errorf("invalid compression type: %q", a)
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be cleaner to treat an empty string as a valid value here, rather than completely bypassing validation in Parameters.Validate()? While algorithm is the only field currently, that might change.

Because +optionalstring in Go becomes an empty string when user did not set the field, it might be best to consider it as valid algorithm name in the Go side (CRD validation for the field can still reject it). It is already taken into account elsewhere in this change: inhttpConnectionManagerBuilder.DefaultFilters() you've added switch-default branch which will use gzip for empty algorithm string.

Comment on lines +30 to +227
Algorithm: contour_v1alpha1.BrotliCompression,
}
}

rh, c, done := setup(t, withBrotliCompression)
defer done()

s1 := fixture.NewService("backend").
WithPorts(core_v1.ServicePort{Name: "http", Port: 80})
rh.OnAdd(s1)

hp1 := &contour_v1.HTTPProxy{
ObjectMeta: meta_v1.ObjectMeta{
Name: "simple",
Namespace: s1.Namespace,
},
Spec: contour_v1.HTTPProxySpec{
VirtualHost: &contour_v1.VirtualHost{
Fqdn: "example.com",
},
Routes: []contour_v1.Route{{
Conditions: matchconditions(prefixMatchCondition("/")),
Services: []contour_v1.Service{{
Name: s1.Name,
Port: 80,
}},
}},
},
}
rh.OnAdd(hp1)

httpListener := defaultHTTPListener()
httpListener.FilterChains = envoy_v3.FilterChains(envoy_v3.HTTPConnectionManagerBuilder().
Compression(&contour_v1alpha1.EnvoyCompression{
Algorithm: contour_v1alpha1.BrotliCompression,
}).
RouteConfigName(xdscache_v3.ENVOY_HTTP_LISTENER).
MetricsPrefix(xdscache_v3.ENVOY_HTTP_LISTENER).
AccessLoggers(envoy_v3.FileAccessLogEnvoy(xdscache_v3.DEFAULT_HTTP_ACCESS_LOG, "", nil, contour_v1alpha1.LogLevelInfo)).
DefaultFilters().
Get(),
)

c.Request(listenerType, xdscache_v3.ENVOY_HTTP_LISTENER).Equals(&envoy_service_discovery_v3.DiscoveryResponse{
TypeUrl: listenerType,
Resources: resources(t, httpListener),
})
}

func TestZstdCompression(t *testing.T) {
withZstdCompression := func(conf *xdscache_v3.ListenerConfig) {
conf.Compression = &contour_v1alpha1.EnvoyCompression{
Algorithm: contour_v1alpha1.ZstdCompression,
}
}

rh, c, done := setup(t, withZstdCompression)
defer done()

s1 := fixture.NewService("backend").
WithPorts(core_v1.ServicePort{Name: "http", Port: 80})
rh.OnAdd(s1)

hp1 := &contour_v1.HTTPProxy{
ObjectMeta: meta_v1.ObjectMeta{
Name: "simple",
Namespace: s1.Namespace,
},
Spec: contour_v1.HTTPProxySpec{
VirtualHost: &contour_v1.VirtualHost{
Fqdn: "example.com",
},
Routes: []contour_v1.Route{{
Conditions: matchconditions(prefixMatchCondition("/")),
Services: []contour_v1.Service{{
Name: s1.Name,
Port: 80,
}},
}},
},
}
rh.OnAdd(hp1)

httpListener := defaultHTTPListener()
httpListener.FilterChains = envoy_v3.FilterChains(envoy_v3.HTTPConnectionManagerBuilder().
Compression(&contour_v1alpha1.EnvoyCompression{
Algorithm: contour_v1alpha1.ZstdCompression,
}).
RouteConfigName(xdscache_v3.ENVOY_HTTP_LISTENER).
MetricsPrefix(xdscache_v3.ENVOY_HTTP_LISTENER).
AccessLoggers(envoy_v3.FileAccessLogEnvoy(xdscache_v3.DEFAULT_HTTP_ACCESS_LOG, "", nil, contour_v1alpha1.LogLevelInfo)).
DefaultFilters().
Get(),
)

c.Request(listenerType, xdscache_v3.ENVOY_HTTP_LISTENER).Equals(&envoy_service_discovery_v3.DiscoveryResponse{
TypeUrl: listenerType,
Resources: resources(t, httpListener),
})
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor thing but could be replaced with subtests to shorten it a bit

Suggested change
func TestDefaultCompression(t *testing.T) {
rh, c, done := setup(t)
defer done()
s1 := fixture.NewService("backend").
WithPorts(core_v1.ServicePort{Name: "http", Port: 80})
rh.OnAdd(s1)
hp1 := &contour_v1.HTTPProxy{
ObjectMeta: meta_v1.ObjectMeta{
Name: "simple",
Namespace: s1.Namespace,
},
Spec: contour_v1.HTTPProxySpec{
VirtualHost: &contour_v1.VirtualHost{
Fqdn: "example.com",
},
Routes: []contour_v1.Route{{
Conditions: matchconditions(prefixMatchCondition("/")),
Services: []contour_v1.Service{{
Name: s1.Name,
Port: 80,
}},
}},
},
}
rh.OnAdd(hp1)
httpListener := defaultHTTPListener()
httpListener.FilterChains = envoy_v3.FilterChains(envoy_v3.HTTPConnectionManagerBuilder().
RouteConfigName(xdscache_v3.ENVOY_HTTP_LISTENER).
MetricsPrefix(xdscache_v3.ENVOY_HTTP_LISTENER).
AccessLoggers(envoy_v3.FileAccessLogEnvoy(xdscache_v3.DEFAULT_HTTP_ACCESS_LOG, "", nil, contour_v1alpha1.LogLevelInfo)).
DefaultFilters().
Get(),
)
c.Request(listenerType, xdscache_v3.ENVOY_HTTP_LISTENER).Equals(&envoy_service_discovery_v3.DiscoveryResponse{
TypeUrl: listenerType,
Resources: resources(t, httpListener),
})
}
func TestDisableCompression(t *testing.T) {
withDisableCompression := func(conf *xdscache_v3.ListenerConfig) {
conf.Compression = &contour_v1alpha1.EnvoyCompression{
Algorithm: contour_v1alpha1.DisabledCompression,
}
}
rh, c, done := setup(t, withDisableCompression)
defer done()
s1 := fixture.NewService("backend").
WithPorts(core_v1.ServicePort{Name: "http", Port: 80})
rh.OnAdd(s1)
hp1 := &contour_v1.HTTPProxy{
ObjectMeta: meta_v1.ObjectMeta{
Name: "simple",
Namespace: s1.Namespace,
},
Spec: contour_v1.HTTPProxySpec{
VirtualHost: &contour_v1.VirtualHost{
Fqdn: "example.com",
},
Routes: []contour_v1.Route{{
Conditions: matchconditions(prefixMatchCondition("/")),
Services: []contour_v1.Service{{
Name: s1.Name,
Port: 80,
}},
}},
},
}
rh.OnAdd(hp1)
httpListener := defaultHTTPListener()
httpListener.FilterChains = envoy_v3.FilterChains(envoy_v3.HTTPConnectionManagerBuilder().
Compression(&contour_v1alpha1.EnvoyCompression{
Algorithm: contour_v1alpha1.DisabledCompression,
}).
RouteConfigName(xdscache_v3.ENVOY_HTTP_LISTENER).
MetricsPrefix(xdscache_v3.ENVOY_HTTP_LISTENER).
AccessLoggers(envoy_v3.FileAccessLogEnvoy(xdscache_v3.DEFAULT_HTTP_ACCESS_LOG, "", nil, contour_v1alpha1.LogLevelInfo)).
DefaultFilters().
Get(),
)
c.Request(listenerType, xdscache_v3.ENVOY_HTTP_LISTENER).Equals(&envoy_service_discovery_v3.DiscoveryResponse{
TypeUrl: listenerType,
Resources: resources(t, httpListener),
})
}
func TestBrotliCompression(t *testing.T) {
withBrotliCompression := func(conf *xdscache_v3.ListenerConfig) {
conf.Compression = &contour_v1alpha1.EnvoyCompression{
Algorithm: contour_v1alpha1.BrotliCompression,
}
}
rh, c, done := setup(t, withBrotliCompression)
defer done()
s1 := fixture.NewService("backend").
WithPorts(core_v1.ServicePort{Name: "http", Port: 80})
rh.OnAdd(s1)
hp1 := &contour_v1.HTTPProxy{
ObjectMeta: meta_v1.ObjectMeta{
Name: "simple",
Namespace: s1.Namespace,
},
Spec: contour_v1.HTTPProxySpec{
VirtualHost: &contour_v1.VirtualHost{
Fqdn: "example.com",
},
Routes: []contour_v1.Route{{
Conditions: matchconditions(prefixMatchCondition("/")),
Services: []contour_v1.Service{{
Name: s1.Name,
Port: 80,
}},
}},
},
}
rh.OnAdd(hp1)
httpListener := defaultHTTPListener()
httpListener.FilterChains = envoy_v3.FilterChains(envoy_v3.HTTPConnectionManagerBuilder().
Compression(&contour_v1alpha1.EnvoyCompression{
Algorithm: contour_v1alpha1.BrotliCompression,
}).
RouteConfigName(xdscache_v3.ENVOY_HTTP_LISTENER).
MetricsPrefix(xdscache_v3.ENVOY_HTTP_LISTENER).
AccessLoggers(envoy_v3.FileAccessLogEnvoy(xdscache_v3.DEFAULT_HTTP_ACCESS_LOG, "", nil, contour_v1alpha1.LogLevelInfo)).
DefaultFilters().
Get(),
)
c.Request(listenerType, xdscache_v3.ENVOY_HTTP_LISTENER).Equals(&envoy_service_discovery_v3.DiscoveryResponse{
TypeUrl: listenerType,
Resources: resources(t, httpListener),
})
}
func TestZstdCompression(t *testing.T) {
withZstdCompression := func(conf *xdscache_v3.ListenerConfig) {
conf.Compression = &contour_v1alpha1.EnvoyCompression{
Algorithm: contour_v1alpha1.ZstdCompression,
}
}
rh, c, done := setup(t, withZstdCompression)
defer done()
s1 := fixture.NewService("backend").
WithPorts(core_v1.ServicePort{Name: "http", Port: 80})
rh.OnAdd(s1)
hp1 := &contour_v1.HTTPProxy{
ObjectMeta: meta_v1.ObjectMeta{
Name: "simple",
Namespace: s1.Namespace,
},
Spec: contour_v1.HTTPProxySpec{
VirtualHost: &contour_v1.VirtualHost{
Fqdn: "example.com",
},
Routes: []contour_v1.Route{{
Conditions: matchconditions(prefixMatchCondition("/")),
Services: []contour_v1.Service{{
Name: s1.Name,
Port: 80,
}},
}},
},
}
rh.OnAdd(hp1)
httpListener := defaultHTTPListener()
httpListener.FilterChains = envoy_v3.FilterChains(envoy_v3.HTTPConnectionManagerBuilder().
Compression(&contour_v1alpha1.EnvoyCompression{
Algorithm: contour_v1alpha1.ZstdCompression,
}).
RouteConfigName(xdscache_v3.ENVOY_HTTP_LISTENER).
MetricsPrefix(xdscache_v3.ENVOY_HTTP_LISTENER).
AccessLoggers(envoy_v3.FileAccessLogEnvoy(xdscache_v3.DEFAULT_HTTP_ACCESS_LOG, "", nil, contour_v1alpha1.LogLevelInfo)).
DefaultFilters().
Get(),
)
c.Request(listenerType, xdscache_v3.ENVOY_HTTP_LISTENER).Equals(&envoy_service_discovery_v3.DiscoveryResponse{
TypeUrl: listenerType,
Resources: resources(t, httpListener),
})
}
func TestCompression(t *testing.T) {
tests := map[string]struct {
algorithm contour_v1alpha1.CompressionAlgorithm
want contour_v1alpha1.CompressionAlgorithm
}{
"default": {algorithm: "", want: contour_v1alpha1.GzipCompression},
"disabled": {algorithm: contour_v1alpha1.DisabledCompression, want: contour_v1alpha1.DisabledCompression},
"brotli": {algorithm: contour_v1alpha1.BrotliCompression, want: contour_v1alpha1.BrotliCompression},
"zstd": {algorithm: contour_v1alpha1.ZstdCompression, want: contour_v1alpha1.ZstdCompression},
"gzip": {algorithm: contour_v1alpha1.GzipCompression, want: contour_v1alpha1.GzipCompression},
}
s1 := fixture.NewService("backend").
WithPorts(core_v1.ServicePort{Name: "http", Port: 80})
hp1 := &contour_v1.HTTPProxy{
ObjectMeta: meta_v1.ObjectMeta{
Name: "simple",
Namespace: s1.Namespace,
},
Spec: contour_v1.HTTPProxySpec{
VirtualHost: &contour_v1.VirtualHost{
Fqdn: "example.com",
},
Routes: []contour_v1.Route{{
Conditions: matchconditions(prefixMatchCondition("/")),
Services: []contour_v1.Service{{
Name: s1.Name,
Port: 80,
}},
}},
},
}
for name, tc := range tests {
t.Run(name, func(t *testing.T) {
rh, c, done := setup(t, func(conf *xdscache_v3.ListenerConfig) {
if tc.algorithm != "" {
conf.Compression = &contour_v1alpha1.EnvoyCompression{
Algorithm: tc.algorithm,
}
}
})
defer done()
rh.OnAdd(s1)
rh.OnAdd(hp1)
httpListener := defaultHTTPListener()
httpListener.FilterChains = envoy_v3.FilterChains(envoy_v3.HTTPConnectionManagerBuilder().
Compression(&contour_v1alpha1.EnvoyCompression{
Algorithm: tc.want,
}).
RouteConfigName(xdscache_v3.ENVOY_HTTP_LISTENER).
MetricsPrefix(xdscache_v3.ENVOY_HTTP_LISTENER).
AccessLoggers(envoy_v3.FileAccessLogEnvoy(xdscache_v3.DEFAULT_HTTP_ACCESS_LOG, "", nil, contour_v1alpha1.LogLevelInfo)).
DefaultFilters().
Get(),
)
c.Request(listenerType, xdscache_v3.ENVOY_HTTP_LISTENER).Equals(&envoy_service_discovery_v3.DiscoveryResponse{
TypeUrl: listenerType,
Resources: resources(t, httpListener),
})
})
}
}

"application/grpc-web-text", "application/grpc-web-text+proto", "application/grpc-web-text+thrift",
var compressor proto.Message = &envoy_compression_gzip_compressor_v3.Gzip{}
compressorName := string(contour_v1alpha1.GzipCompression)
if b.compression != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that DefaultFilters() function used to set up a set of fixed filters - I don't see any of the other builder method impacting the default set except the new Compression() method, which now also makes it important in which order the builder methods are called - Now the configuration would be different depending if Compression() is called before or after DefaultFilters().

Maybe it would be better to (1) remove compressor from the default set completely and only add it when the new Compression() builder method is called? Or maybe some other alternative: (2) instantiate gzip by default and modify it later according to chosen algorithm when Get() is called or (3) delay adding compressor until Get() but add it by default at that point.

| debug | boolean | `false` | Enables debug logging. |
| default-http-versions | string array | <code style="white-space:nowrap">HTTP/1.1</code> <br> <code style="white-space:nowrap">HTTP/2</code> | This array specifies the HTTP versions that Contour should program Envoy to serve. HTTP versions are specified as strings of the form "HTTP/x", where "x" represents the version number. |
| disableAllowChunkedLength | boolean | `false` | If this field is true, Contour will disable the RFC-compliant Envoy behavior to strip the `Content-Length` header if `Transfer-Encoding: chunked` is also set. This is an emergency off-switch to revert back to Envoy's default behavior in case of failures. |
| compression | string | `gzip` | Sets the compression type applied in the compression HTTP filter of the default Listener filters. Setting this to `disabled` will make Envoy skip "Accept-Encoding: gzip,deflate" request header and always return uncompressed response. Values:`gzip` (default), `brotli`, `zstd`, `disabled`. |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compression here is still string while it should be CompressionAlgorithm. The new type needs a new table with algorithm field with the content described here.

@@ -0,0 +1 @@
Add "compression" object to define settings in default HTTP filters, initially just supporting changing/disabling compression algorithm.
Copy link
Member

@tsaarni tsaarni Nov 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could add some pointers here for the user how to use it, possibly for example:

Suggested change
Add "compression" object to define settings in default HTTP filters, initially just supporting changing/disabling compression algorithm.
The HTTP compression algorithm can now be configured using the `compression.algorithm` field in the configuration file or the `spec.envoy.listener.compression.algorithm` field in the `ContourConfiguration` CRD. The available values are `gzip` (default), `brotli`, `zstd`, and `disabled`.

@geomacy
Copy link

geomacy commented Nov 22, 2024

Many thanks for the comments @tsaarni will have a look through and apply changes as soon as possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note/small A small change that needs one line of explanation in the release notes.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants